No-reference Video Quality Assessment and Applications

With more and more visual signals being received by human observers, an important aspect of the quality of experience of such stimuli is the perceived visual quality. In this thesis, new techiques to assess this perceived visual quality of natural videos without a pristine reference video, referred to as no-reference video quality assessment (NR-VQA), are presented, in order to evaluate the performance of existing devices for video capturing or video compression. These techniques adopt a two-stage NR-VQA framework, in which the two stages are distortion measurement and quality prediction. Three NR-VQA metrics are designed to evaluate the performance of video imaging systems, while two computational NR-VQA models are proposed to assess the quality of compressed videos. An optimizing strategy is also designed for feature pooling and prediction models of NR-VQA algorithms. Loss of texture is one particularly annoying distortion in videos captured by consumerlevel camcorders. Three NR-VQA metrics, viz., texture preservation ratio (TPR), physical texture distortion (PhTD), and perceived texture distortion (PeTD), are designed to evaluate the performance of a camcorder system in terms of compression bitrates and the speed of the object’s motion. The three metrics first quantify to what extent texture structures are distorted, then a perceptual weighting strategy is adopted for quality prediction. Assuming the distortion is radially symmetric, the TPR is based on 1-D radial power spectral density (PSD) and 1-D contrast sensitivity function of the human visual system, while PhTD and PeTD measure radially asymmetric texture distortion based on 2-D PSD and a 2-D spatio-velocity contrast sensitivity function. Experimental results of six camcorders confirm that all three metrics are useful indicators for evaluating a camcorder’s performance. Most artifacts encountered in a video are a direct result of lossy compression, thus predicting the quality of compressed videos is of great interest. Two NR-VQA models are proposed: a DCT-based model, and a Laplacian-pyramid based model. In each model, six features are extracted from each frame of a video sequence to measure the distortion, considering the statistical features of natural videos and the extraneous artifacts due to lossy compression. In the stage of quality prediction, both models adopt a temporal pooling strategy to transform frame-level features to video-level features, and a trained multilayer neural network to map the video-level features to a score as the predicted video quality. Both models are trained and evaluated on the LIVE video database and the LIVE mobile video database. The results show that the predicted quality scores are well correlated with the mean opinion scores from subjective assessments. However, the neural-network based quality prediction has many parameters, and thus is prone to overfitting when the training database is small. In order to circumvent the intractable problem of overfitting, a mapping strategy is proposed to optimize the feature pooling and prediction models of NR-VQA algorithms. Based on iterative optimization, the proposed mapping strategy is composed of a global linear model for pooling extracted features, a simple linear model for local alignment in which local factors depend on source videos, and a non-linear model for quality calibration. In the IRCCyN/IVC video database of content influence and the LIVE mobile video database, the performance of VQA algorithms is improved significantly by local alignment. This suggests that predicting the local factors for a local alignment based on video content will be a promising new approach for NR-VQA.